fix(mcp): configure cross-encoder from providers instead of hardcoded OpenAI by winklemad · Pull Request #1637 · getzep/graphiti

winklemad · 2026-07-08T16:41:26Z

Summary

The MCP server did not pass a cross_encoder to Graphiti(), so Graphiti defaulted to OpenAIRerankerClient() and required OPENAI_API_KEY at startup even when the configured LLM and embedder both used other providers. This fixes #1393 without allowing a failed local fallback to silently restore that OpenAI default.

Type of Change

Bug fix
New feature
Performance improvement
Documentation/Tests

Objective

Select an available reranker from the configured providers and keep non-OpenAI MCP configurations independent of an OpenAI key.

Change

Add CrossEncoderFactory in services/factories.py. It prefers the LLM provider, then the embedder provider, and uses Graphiti's local BGERerankerClient when neither provider has a native reranker.
Pass the selected cross-encoder into both FalkorDB and Neo4j Graphiti() construction.
Warn that the local BAAI/bge-reranker-v2-m3 fallback may download about 2.3 GB on first run.
If the base MCP install lacks sentence-transformers, fail startup with actionable providers-extra / Graphiti-extra / provider guidance. The error is intentionally not swallowed into cross_encoder=None, which would silently restore the OpenAI reranker.

Testing

Unit tests added/updated
Integration tests added/updated (not required for factory selection/error propagation)
All relevant existing tests pass

Validation run locally:

pytest tests/test_cross_encoder_factory.py -q — 4 passed
pytest tests/test_factories.py tests/test_cross_encoder_factory.py -q — 28 passed
ruff check . — passed
ruff format --check . — passed
Focused Pyright on the changed files — passed

Breaking Changes

This PR contains breaking changes

Checklist

Code follows project style guidelines (make lint equivalent checks pass for the changed files)
Self-review completed
Documentation updated where necessary (no public configuration schema change)
No secrets or sensitive information committed

Related Issues

Closes #1393

… OpenAI The MCP server never passed a cross_encoder to Graphiti(), so it fell back to OpenAIRerankerClient and required OPENAI_API_KEY even on fully non-OpenAI setups. Add a CrossEncoderFactory that picks a reranker from the LLM provider, then the embedder provider, and falls back to the local BGE reranker, and wire it into both Graphiti() construction sites. Fixes getzep#1393

zep-cla-assistant · 2026-07-08T16:41:39Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

winklemad · 2026-07-10T14:28:18Z

I have read the CLA Document and I hereby sign the CLA

winklemad · 2026-07-14T19:07:02Z

I have read the CLA Document and I hereby sign the CLA behalf on myself, e-mail: winklemad@outlook.com

winklemad · 2026-07-17T22:55:34Z

I have read the CLA Document and I hereby sign the CLA behalf on myself, e-mail: winklemad@outlook.com

Naseem77 · 2026-07-21T06:52:43Z

@winklemad Tested this on a clean clone: the MCP cross-encoder used to always be OpenAI regardless of config, which crashes if you only have a Gemini or Anthropic key. With your patch the factory returns the right reranker per provider and falls back to BGE local. Ran the PR tests plus provider-combo tests, all pass.

winklemad · 2026-07-21T11:28:58Z

Thanks @Naseem77 — appreciate the clean-clone test, and good to have the FalkorDB reranker path independently confirmed.

Status for maintainers, in case it helps: I've signed the CLA (winklemad@outlook.com) and Ruff is green — the remaining Required checks (CLA re-check, pyright, unit-tests) are all just waiting on first-time-contributor workflow approval rather than any actual failure. The branch is behind main but merges cleanly; happy to rebase if that's easier. Whenever someone can approve the CI run or take a look, I'll address any feedback quickly.

abouchard11

Tested this on my production stack, complementary to @Naseem77's clean-clone FalkorDB run: Neo4j 5.26 + Gemini LLM (gemini-2.5-flash) + gemini-embedding-001, config loaded through the server's actual GraphitiConfig loader. The factory selected GeminiRerankerClient and a live rank() call against the Gemini API returned correct ordering. test_cross_encoder_factory.py + test_factories.py: 26/26 pass locally.

Two findings on the fallback path, with a suggestion inline:

On a base install (no providers extra), sentence-transformers is missing, so when neither provider has a native reranker (e.g. Anthropic LLM + Voyage embedder) the BGE import raises ImportError. graphiti_mcp_server.py catches it, logs a warning, and passes cross_encoder=None — and graphiti-core then defaults to OpenAIRerankerClient(), which is exactly the #1393 crash this PR fixes, now hidden behind a misleading warning. Reproduced by blocking sentence_transformers in the venv. The suggestion below at least makes the failure actionable; whether reranker-init failure should be fatal is a maintainer call.
BGERerankerClient() instantiates CrossEncoder('BAAI/bge-reranker-v2-m3') in its constructor — a ~2.3 GB synchronous download at server startup the first time the fallback runs. Folded a heads-up into the suggested log line; probably worth a README note for Docker users too.

This supersedes my #1669 (Gemini-only take on the same idea), so I'm closing that in favor of this one. Also happy to pick up the explicit reranker: config block you floated in the description as a follow-up PR once this lands.

abouchard11 · 2026-07-22T12:03:29Z

+        logger.info('No provider reranker available, using local BGERerankerClient')
+        from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient
+
+        return BGERerankerClient()


Makes the failure actionable when sentence-transformers isn't installed, and flags the first-run download:

Suggested change

logger.info('No provider reranker available, using local BGERerankerClient')

from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient

return BGERerankerClient()

logger.info(

'No provider reranker available, using local BGERerankerClient '

'(downloads BAAI/bge-reranker-v2-m3, ~2.3 GB, on first run)'

)

try:

from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient

except ImportError as e:

raise ValueError(

'No provider reranker is available for this configuration, and the local '

'BGE fallback requires the optional sentence-transformers dependency. '

"Install the MCP server's 'providers' extra (uv sync --extra providers) "

'or configure an OpenAI or Gemini key so a provider reranker can be used.'

) from e

return BGERerankerClient()

Naseem77 mentioned this pull request Jul 20, 2026

[Graphiti] Identify FalkorDB-relevant changes in upstream Graphiti PRs FalkorDB/graphiti#22

Open

Naseem77 mentioned this pull request Jul 22, 2026

feat(mcp): use a provider-matched cross-encoder to avoid a hard OpenAI dependency #1669

Closed

abouchard11 reviewed Jul 22, 2026

View reviewed changes

fix(mcp): fail when local reranker is unavailable

8cf8cdb

zep-cla-assistant Bot added a commit that referenced this pull request Jul 22, 2026

@winklemad has signed the CLA in #1637

f5cf233

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(mcp): configure cross-encoder from providers instead of hardcoded OpenAI#1637

fix(mcp): configure cross-encoder from providers instead of hardcoded OpenAI#1637
winklemad wants to merge 2 commits into
getzep:mainfrom
winklemad:fix/mcp-configurable-cross-encoder

winklemad commented Jul 8, 2026 •

edited

Loading

Uh oh!

zep-cla-assistant Bot commented Jul 8, 2026 •

edited

Loading

Uh oh!

winklemad commented Jul 10, 2026

Uh oh!

winklemad commented Jul 14, 2026

Uh oh!

winklemad commented Jul 17, 2026

Uh oh!

Naseem77 commented Jul 21, 2026

Uh oh!

winklemad commented Jul 21, 2026

Uh oh!

abouchard11 left a comment

Uh oh!

abouchard11 Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

-        logger.info('No provider reranker available, using local BGERerankerClient')
-        from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient
-        return BGERerankerClient()
+        logger.info(
+            'No provider reranker available, using local BGERerankerClient '
+            '(downloads BAAI/bge-reranker-v2-m3, ~2.3 GB, on first run)'
+        )
+        try:
+            from graphiti_core.cross_encoder.bge_reranker_client import BGERerankerClient
+        except ImportError as e:
+            raise ValueError(
+                'No provider reranker is available for this configuration, and the local '
+                'BGE fallback requires the optional sentence-transformers dependency. '
+                "Install the MCP server's 'providers' extra (uv sync --extra providers) "
+                'or configure an OpenAI or Gemini key so a provider reranker can be used.'
+            ) from e
+        return BGERerankerClient()

Uh oh!

Conversation

winklemad commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Objective

Change

Testing

Breaking Changes

Checklist

Related Issues

Uh oh!

zep-cla-assistant Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

winklemad commented Jul 10, 2026

Uh oh!

winklemad commented Jul 14, 2026

Uh oh!

winklemad commented Jul 17, 2026

Uh oh!

Naseem77 commented Jul 21, 2026

Uh oh!

winklemad commented Jul 21, 2026

Uh oh!

abouchard11 left a comment

Choose a reason for hiding this comment

Uh oh!

abouchard11 Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

winklemad commented Jul 8, 2026 •

edited

Loading

zep-cla-assistant Bot commented Jul 8, 2026 •

edited

Loading